here are sessions Ellen recorded (others recorded by Ainsely)
Core Architecture of the Business of Retrospectives* Wednesday, April 10. 2002, Sylvia Beach Hotel, Newport, Oregon Workshop for Retrospective Facilitators
*Assumes we have completed contracting; we are at the post-qualification stage (we have qualified the client and have the gig)
Convener: Linda Participants: Njke, Esther, Ellen, Gerhard, Janis, Diana, Dan, III, Diana, Michael
Questions we want to answer (assume we have a client): What are the elements of the package? What do you charge for the package? What is “free” in the package? e.g: phone calls – see notes at the end of this document Charging: (one of us charges for three days of retrospective facilitation irregardless of what it takes). Phases: before (pre), during and after (post)
Stages, steps, elements:
Pre-retrospective: note1: some of the things may change depending on the client. Note2: these are not listed in order. o Prepare and send out questionnaires (maybe sent via email) o Ask for project effort data, if you are collecting it o Conduct face-to-face interviews (depending on the situation) o Tell attendees that all this is in confidence; we would want to present results in a summary form to the whole group, so tell me if you have something to tell that you want to remain in confidence o On-site visits. If we have to travel cross-country, we might go 2-3 days early or have to make separate trips to have a site visit before the session. We would want to build this into the fee. o May want to meet with a few key people and “grog” the culture o Questions may be the same but the format of getting them answered may change. o Customers don’t often understand that the pre-work and workshop design takes a lot of time. Thus, we may build it into the charge all the pre-work o Roughly a day of preparation for a day of workshop o Design the session (flow of activities and their timing) o The lapse time might spread up to 8 weeks o Venue logistics are also part of the pre-work: location, supplies, tools, food o Training retrospective interviewers (as applicable) o Invitations: we might have a letter but internal folks send out an email o Deciding who should be invited o Work with inviter to get the people there o May want to work with a meeting planning o May need facilities with capabilities for overheads, laptop projectors, microphones, printer, videotapes, pin walls, sticky walls o Some checklists are now available on Ellen’s site related to her book (www.ebgconsulting.com) which are generically useable for not just requirements but also retrospective workshops o Coordinating with a co-facilitator if there is one o Coordinating with the recorder if there is one o Ground rules – basic ones like interruptions o Knowing how to handle messages etc. – getting agreement to have no interruptions or a “runner” (someone to run after an attendee if there is a critical problem that requires a participant’s attention), if they are really anal o May want to establish a planning or design team (the time they need to help the facilitator plan and design and arrange participants etc. are actually part of the customer’s cost)
During o Materials cost (markers, index cards, etc.) o Remember to use dark colors on your flips o Watch out for allergies from scented markers! o Use lower case and not caps: per research o Or, use upper case to make them slow down and read o Use graphics on your posters for fun and right brain stuff o Facilitation itself (run the workshop) o Markers: www.neuland.au or www. Neuland.at (Neuland No. One – refillable) were markers shared by Gerhart that many of us fell in love with. There was almost orgasmic excitement with these markers! You can also go to www.artsupplies.com to buy dark markers; and to www.grove.com for excellent markers, Charter’s Markers) o How long are you there, e.g. do you do work in the evening? depends on if you live in the city of the session o One or more facilitators: may have a co-facilitator for larger groups or complex projects o Some clients want to use their own co-facilitator – may want to “qualify” him or her o Colored crayons that are blocked shaped to highlight things on a poster: contact Gerhart and he can give us these; we also got excited about these~!!!! o Post-it and index cards: powerthought.com (glue is one side) and www.vis-it.com has the 3M glue on both sides for post- Dots by Alvin “professional dots” for posting cards on the wall o Butcher paper or rolls of paper o Spray: use the 3M brand! it (see Ellen’s site has the 3M item number to order for the 3M spray can that works best) o Rip-stop nylon for the wall. Can reuse them; fold them not on the sticky side. Tape them up with tension to undo any wrinkling problems. o Dry erase posters and dry erase markers can also be used, but be careful about erasing onto the wall! o Infrared recorder that captures the stuff (less success with those, e.g. e-beam or mimio) o Digital camera for capturing the walls and posters – very useful o Flipcharting: The Big Book of Flip Charts is a great book and also The Grove has a booklet (and training) on graphic images and graphic facilitation with tips on how to draw images of complex topics o Need to have an activity in the workshop for how the information will get carried forward (which leads into the next point) o Recommendations and actions planning
After o May write up a report o Tip: Put big issues on the first two pages of the report o May record the flip charts, photographs o Reports and analysis – my view of what happened o May just have a photo protocol (e.g. a document of photos) o Price will be different if they want a formal document which is a timeline of what happened in the workshop o Write-ups are valuable to the client per Gerhard: they use it years latter. The sponsor gets it and is responsible to distributing it, deciding if is confidential or if not, who it should be shared with o Question: if we use material from a retrospective, can we use it in other forms: Get permission to used ‘masked’ info in a way such as Linda for pattern seeking May put in our contract that would want to use the ‘story’ and masking it; some clients may say no! Can’t NOT use the knowledge and experience gained in our work and in our writing o Debrief yourself. We each re-think the retrospective we facilitated to find we worked, didn’t etc. (not part of the pricing!) o Do you charge for process-improvement? No, don’t call it out on the package price o Follow-up phone calls to check in on progress – holding accountable which is agreed upon ahead of time o Follow-up to think about leveraging for future work, what used, didn’t use and why and why not (not part of the package) (essential for our own ongoing debriefing and learning)
Pricing: o Travel is outside the package price o Reason for package price: the client can’t pick and choice things like the “during” portion only o Customer’s cost: people to help with the planning and design o How much do you charge: Pricing varied quite a bit in the group: For up to 3 days of retrospective: $10K for one facilitator plus Time and Expense (agreement with Norm); some felt that was too low May want to get waivers for fees like getting “vendor” approval, so maybe need to update $1750 per day assuming 3 days $10-12K normal rate $1800-2K rate per day normal rate $2500-3500 rate per day
Some early discussion issues:
“Free” phone calls: • Include the free stuff in your total price • Limit the amount of time on free phone calls • Make it visible • Have a reasonableness clause for calls • Call when my body is somewhere else I don’t want it to be but if the calls get to heavy I say: it’s time for another visit.
Wanting to “bow out” (not do the engagement): • Tell the truth and get out of the gig • Add a clause to renegotiate if circumstances changes in the contract • Add a cancellation clause in the contract
Meta-level Approaches to Focusing in Groups Wednesday, April 9, 2002, Sylvia Beach Hotel, Newport, Oregon Workshop for Retrospective Facilitators
Convener: Frowin Participants: Bent, Norm, Diana, Michael
1) What is the problem? 2) Brainstorm to collect symptoms - how did it happen? 3) Which influences can we not change? 4) Ideas for solutions - brainstorm 5) Which ideas identified in step 4, might actually work? 6) Concrete action items - who does what when?
1) mess finding (look for areas of un-ease) 2) data finding (based on step 1, background circumstances) 3) problem finding (who=s problem is it? clearly define it) 4) idea finding (alternatives) 5) solution finding (candidate set of possible remedies, develop evaluation criteria) 6) acceptance finding (ongoing adherence, visualize ideal situation and the terror of not doing anything at all, reflect on potential problems coming from proposed solutions)
AWho wants to be a millionaire@ question: ask everybody what needs to be done - if nearly all agree, then they=re nearly always correct
Topics: How to find the topics to work on in groups How to bring people into a productive mood
Sources of Unease: wanting to do the Aright@ thing getting deep enough (e.g., one day retrospectives present a challenge) sorting out the diversity of agendas, ideas, etc., to get to the CORE issues and achieve a clear sense of direction do no harm - it=s possible to create a false sense of safety that could get people hurt do I have the right people in the room? wanting to fix Athe soup@ (knowing what we cannot change) We want to enable participants to reach the ground of their problems/issues/challenges - what do the participants want?
safety and safe conditions
ask the sponsor - what do I do if the participants want to talk about other things? (other than the sponsor wants them to talk about)
qualifying the client self-aware? are they willing to do three days (at least the first time)? are they experienced with retrospectives? do they support the values of the Prime Directive?
possible marketing angle: All teams can do a one-day retrospective. Novice teams (new to retrospecting), however, require two additional days of preparation time.
Time spent together builds the Aright mood@ - stretching the given time.
influencing AThe Chiefs@ - and their role in prioritizing
In response to symptoms (hunches or gut feelings) that we are not working on the right things: is their any benefit in continuing? how much trust do we have from our sponsor (to Aburst bubbles@)
Symptoms we are not working on the right stuff: repetitive conversations and repeating phrases Agush@ (shorthand for I strongly agree with what was just said) increase structure talking stick dialogue three cards (e.g., yellow = I have something to say, green = I agree, red = I disagree - enables participation without talking) visualization silence low energy levels serial broadcasts (speaking without listening) being Atoo nice@ confront niceness out Anice@ them (risks the appearance of making fun of them and being counterproductive, probably inappropriate for large groups) name the effect of niceness hopelessness we have three choices: love it, leave it, or change it
asking for what you want increases your odds of getting it:
yes no
ask for what you want 0 - 100% 0 - 100%
don=t ask for what you want 0% 100%
———-
What did Gandhi do?
1. He knew when to duck. 2. Met everyone with mutual respect. 3. Evangelized but avoided fanaticism. 4. Allowed followers to choose to follow. 5. Dealt from the truth. 6. Completely open about what he was doing. 7. Neutralized the effect of his opponents’ weapons. 8. Used his opponents’ laws to neutralize them. 9. Respected everyone. 10. Comfortable with all creeds. 11. Willing to personally do what he asked of others. 12. Practiced beforehand; anticipated consequences. 13. Knew what moved folks he wanted to change; he could sing their songs. 14. Grounded in simple principles. 15. Demonstrated passion. 16. Remained true to his mission; continually referred to it. 17. Allowed folks to signify their commitment. 18. Stressed commonality with adversaries; helped them do the right thing. 19. Met resistance congruently; acknowledged self/other/context. 20. Deployed extraordinary courage—it wasn’t a choice, it just was. 21. Communicated: locally, globally. 22. Good press provided external witness to his story. 23. Accepted accountability for his actions. 24. Made himself touchable.
Leading without power – these characteristics are all learnable
1. Vision 2. Persistence 3. Confidence 4. Optimism
Improvisation, simulations and role plays in retrospectives.
Art Gallery: draw your impression of this project. Can do as you saw it, and as you wish. (action plan the gap)
Picture of the project
Slogan Adjectives
«image doesn't appear: box labeled on left top w Pictuer of the project, bottom right: slogan and right side: adjectives«
Build the project with boxes, tape ect for an unfinished project
Tell a story/build a story of the project. Can begin with “once upon a time”
Role play a scene that would tough from the project
Build a list of artifacts that you wish were there
Do sculpting of an event in the project. No talking. Freeze and check in with the players
Use colored posts or pens to write down events that happened; each color represents your reaction to that event
Tower of babel: ??
3 headed expert to debrief the group and have fun
cooperation machine
Perspectives Thursday, April 12, 2002, Sylvia Beach Hotel, Newport, Oregon Workshop for Retrospective Facilitators
Convener: III Participants: Njke, Esther, Ellen, Bent, Janis, Diana, Linda, Pat, Michael, Gudran, Siegie, Debra, Gerhard
Premise of the Session:
Charters are a way to start a project (keeping the plan out of it): a document to expectations; a way to define how to use someone else’s resources. Should set up in a charter what to measure. One of the most precious of human experiences is failure. Failures give us the poke to change. Having a charter to start with is tool to compare against during a retrospective.
In other words:
A retrospective is a charter’s way of having another charter. A charter is a retrospectives way of having another retrospective. Closing the loop; looping the close.
“Boundary is everything”. Other words: scope, context.
These things feed the charter: Vision: view of the future, lasting (e.g. word peace) Mission: direction to choose to follow-up on the vision (e.g feed everyone, give people political choice) Strategies: choices you make when you make them All guided by principles: statements of value that guide and reward desired behavior (e.g. the Declaration of Independence: “we hold these truths to be self evident”)
Charter content (III’s recommendations):
Objectives: a statement of external, measurable effect – SMART (specific, measurable, realistic and time-based)
Boundary exhibits: • Context diagram • Key event list
Committed resources: Money, people time, tools, training, access, environment, permission to iterative (the gold owners (people who are spending the money) should say how much they are willing to gamble)
Authorizing players: People who 1) can testify if the work today is okay and 2) answer: “may we continue?”
III thinks that for every person hour spent doing chartering; you will save a person month latter. How do you do this when people have been dispersed?
Other things people do in charters: key interactions, risks, assumptions, related projects, etc.
We shifted back discussion to the implications of charter-centric retrospectives (or retrospective-centered charters!!): • Makes the retrospective more measurable • (Measures for products vs. services are different. Product measures are specific and service measures are things around perception, feeling of satisfaction, ease of use survey, etc.) • Charters become better over time • Get to understand the forces behind the gap in between the intended charter and the actual result – if variances occur between objectives and outcome can seek why • More likely to continue to doing chartering • People become educated to think about thinking first, then acting • Becomes a decision filter • Gets gold owners (sponsors) more aware of what it takes to do a system: SMOP – small matter of programming (Lawrence Livermore Lab term – they realized that 10% of the work was coding and 70% was chasing down what to do and 20% was organizing the technology) • Satisfaction (of reaching what you wanted to do) – it feels good to succeed
Implications of retrospective-centered chartering: • Begin with the end in mind • Slims down the charter (take less chunks, do more frequent retrospectives) • More likely to keep the same group of people who work well together to stay together • Openly doing subliminal advertising • Makes people more likely to have an effective retrospective (keep journals, artifacts, “lullaby” notes) • Self-fulfilling (positive) prophesy • More likely to keep the charter alive and updated – may inspire formal charter revision • End the retrospective with a plan to do a charter for the next charter • More likely to have sponsorship for the retrospective and sponsors be AT the retrospective, since you will have sponsors in the charter
Cautions of doing it: • You fall in love with your structure – rigidity; being in love with the pieces • Feel like all you’re doing is workshops, not the real work! • “Go away” – people tell me they don’t want me around because “I don’t have time for this crap”. • By melding the two together, the full effect of doing the two as separate things gets lost – might blur the full completion of retrospectives • If combine them, they will want both to be done at the same time • May start to clone old charters
Book recommendation: The systems approach and it’s enemies by C. West Churchman: a book about boundaries…
Quotes on Failure:
“A failure is a man who has blundered but is not capable of cashing in on the experience.” -– Elbert Hubbard
“The word LOSE simply stands for Lack Of Success Expectancy.” -– Doug Firebaugh
“Failure is a prerequisite for great success. If you want to succeed faster, double your rate of failure.” -– Brian Tracy
“The only time you can't afford to fail is the last time you try.” -– Charles Kettering
———
Article Ideas - The Fœnix Group Wednesday, April 9, 2002, Sylvia Beach Hotel, Newport, Oregon Workshop for Retrospective Facilitators
Thursday Home Group session on Article Ideas Convener: Esther Home Group: Frowin, Bent, Nynka, Sue, Michael
Seven (plus or minus X) Reasons Not to do a Retrospective:
o you might learn something you don=t want to know o you might find out you are part of the problem o you might find out people don=t like your behavior (or that they do) o you might find out how people really feel o you might find yourself in unfamiliar territory o there is a slim chance it might cost more than it gains o might do more harm than good, particularly if you don=t follow up and follow through o it may land you in a Aguilt trip@ o you may find you need to make some changes o you may find it=s not all their fault o you may find it=s not all your fault o you may lose the illusion of being in control o you may actually learn something new (or old) o your people may turn Apsycho@ o you may find out there are not only facts, but feelings as well o you may become incongruent o people may start (or stop) talking to each other
o you might receive bad news earlier (in time for intervention) o you might expose yourself to legal liabilities
———– Appreciative Inquiry
Diana lead the session on what to do when a team slips into blaming each other. She discussed what to do when a team are all feeling this wasn’t a failure but not a huge success and don’t gain anything by participating in the retrospective.
Techniques discussed:
Temperature Reading: from Virginia Satir. Use this technique at the end of the first day or when a group doesn’t know where they are. Start with Appreciations and move through the other key items. • Appreciations • Puzzles • Apology • Complaints/Recommendations • New Information • Hopes & Wishes
Thermometer (symbolic): an anonymous safety check that you can use everyday at the beginning and end.
Wall of Wonder: Used with groups where you feel safety is not an issue where you want to uncover assumptions. Answer the question: Why is this project in this situation? Then pause and ask the question that deepens the inquiry. Write one per card what you learned and put up on the wall.
Prouds and Sorries or Mads Glads and Sads: Use index cards for participants to answer the question: What you the most proud of? What are you the most sorry about? Cluster the cards on the wall and talk about what is there
When two (or more) parties disagree: Name it, focus on content, and deal with the individuals. Remember, people of goodwill can disagree. OVP – take on another point of view, and perspective wheel where you look at different slices of pie (answer questions) as a developer, project sponsor, and tester.
Visitor from Outer Space: Areas of measure, phenomena/evidence, specific measures, choose top ten list.
What do the stakeholders value? Who are stakeholders, specifically by name? What would you do what do they value about x? How can we provide more continuance?
Evolutionary Project management: EVO is a technique used to bring out who are the stakeholders are and what value can you bring to them within a few weeks?
Horror Stories: Used to get a team to picture what happened during a project. Have them individually write a “title of the story”. This is to be used as a proactive exercise.
Globally Dispersed Groups
Lunch discussion centered around the following:
• Use own language to deliver training and the participants used their own language to answer the questions in their own language • Use 3×5 cards to express thoughts anonymously • Slow down when speaking English • Think about semantics • Remember different cultures • Very tiring to deliver training in more than one language
Improv and Simulation
Mind mapping discussion identified a loose agenda:
• Warm-up • Definitions • Shared experiences, methods/techniques, stories • When and why do we use these techniques? • Invent a simulation (Debriefing, players, roles, length) • Resources (books, web, attitudes, cards) • Wrap-up
Warm-up:
Ellen led the group in Slap, Clap and Snap as a warm-up exercise
Definitions:
Improv – activities with players where a scene is set and rules are discussed Simulation – acted out, metaphor that represents something real Role-playing and game were also discussed
Shared experiences:
Siegi shared pictures of her sculpture using boxes. The best part was pushing the boxes out of the 3rd story window.
Esther, Nyka, and Bent showed the group what “Sculpturing” was all about by sharing an experience they shared. This technique requires no talking, just acting out the emotions of the players without a script.
Gerhard shared a technique he uses where he splits up a flipchart into three sections and asks teams to: Draw a picture of how the ideal project would look and another group draws reality. He has a section of adjectives that describe the picture and they end with a slogan. He shared the teams typically draw a ship with the metaphor of a captain, they go in a direction, they have waves they have to fight through and if they don’t get them resolved they sink.
Ellen shared a technique called “Cooperation Machine”. She asked one person to come up and begin doing a repetitive motion. Then she invited another person to get in synch and another person to add to the “machine”.
Next Ellen asked someone to come up and try the “Un-cooperative Machine”. This time she added another person who worked against the others and she added a third person. We debriefed the experience.
Pat shared her company used tinker toys to build a tower. No talking and no sharing of resources. They measured success by the amount of time it took to make the tower that matches the specs of the customer. The next day the teams transform because they can now talk and share resources. They are now able to reduce the time they need and the tower reflects exactly what the customer needs.
When and why do we use these techniques?
• When safety is high with the group • Use them sparingly • When you want to touch on deeper issues • To get below the surface level discussions • Allow to get a different perspective • To disturb the mental model • Ask questions to bring out feelings (happy/sad)
Invent a simulation:
Ellen was interested in working on a dinner theme with roles (family) cooking dinner. If anyone is interested in inventing this simulation she asked we contact her.
Resources:
Ellen was using a book titled “Playing Along” by Gesell. This book has 37 games and debriefing questions.
Ellen also mentioned a book titled “Bird by Bird” by Annie McDermitt (Spelling?)
Six Hats, allows a different mode of thinking. Black hat is the judge, red is emotion, yellow is sunny, blue is facilitator… One suggestion is not to use all six hats for an exercise.
Whack Pack and Whack on the side of the head by Roger von Oech
Other books by the title: Games Trainers Play
Debra mentioned she has used “Broken Squares” a cooperation exercise.
Esther shared a Meta debrief technique ORID: Observational, Reflective, Interpretive, And Decisional. This is from the “Art of Conversation”.
Other resources: www.thiagi.com www.deepfun.com www.nasaga.org
Wrap-up:
Ellen summarized the session with a technique called: Three Headed Expert. This technique has three people sitting side by side and they answer questions from the group one word at a time. Each of the three people adding a word to the answer. When the last person believes the question has been answered they go to the next question by saying: “NNNNN-ext Question! In unison. It was humorous way to get the group to close the session.
Internal Retrospectives
Debra lead two sessions on Internal Retrospectives. The people that attended this session and the organizations they came from:
• Pat: Lucent • Bent: Danish Savings Bank • Debra: Intel Corporation • Frowin, Gudrun, Siegi: Siemens
The group talked about the different names the internal groups call retrospectives:
• Post Project Reviews • Project Experience Workshop • Project Evaluations • Phase Review • Retrospectives
The group identified the following agenda to discuss our internal retrospective projects:
• Concept • Design • Implementation • Sustain
Discussions concerning the concept of doing retrospectives:
Everyone shared how retrospectives got started in their organizations:
What triggered the start of retrospectives in their organizations? PAIN, managers interested in learning to improve their part of the development, managers wanted to be considered “special”, developers asked for a retrospective.
15-16 years ago Pat’s group were assigned to look at ways to improve quality…doing “post mortems” already, 3 versions of the product were already – this was a continuing large project.
A project failed, so they asked “What can we do to learn?” Norm was a resource to help Gerhard get a large project (300 people) to try out a new phased type approach – 5+ years ago. Key good contacts management. Management was open to experimentation. It is imperative that management supports retrospectives, you need a sponsor for them. Frowin said Seimans charges for their services.
System Arch Review Board (SARB) was set up within Lucent and has been part of the review process for 12 years. Expanded reviews to include retrospectives. Pat said they charge for their services.
Bent shared how his group got started with retrospectives. He said in 1980 he collaborated with another co-worker to create a developer training four day workshop. He said the key to success was to meet with projects often, early, and informally.
His group was called the “Wishful Thinking” department. This group grew in 10 years to 12 people and then shrank down to two people in 1990.
Bent drew a timeline of a project to overlay what he has seen as a trend for the teams he has worked with. This timeline indicated two times that he feels is best to work with teams: Early and late in the project.
At a kickoff meeting it was suggested we ask those teams that don’t have a history to inquire with members and ask them what went well, what they don’t ever want to do again, a guessing session of what might go wrong? Include a risk analysis to talk about cost and have the managers attend the end of the session to get the final results.
It was discussed the possibility of holding a retrospective during smaller phases of the project to stop repetitive problems, early before the project concludes. For long projects, it is easier to make course corrections during different phases of the project. Include key people, everyone on the team is invited to the retrospective, this is usually a 2-4 hour retrospective.
The last time it is optimal to do a retrospective is at the end of the project, just after it ends, but before the next project begins.
Discussions concerning design of a retrospective:
Workshop life cycle of Siemens retrospective program: • Contact (know who is paying for the workshop, who is responsible for what will happen in the workshop • Contracting (made by facilitator who will deliver the workshop to determine a date to deliver the workshop, price, objectives/goal, team dynamics, bad outcome, checklist, excluded items, ask the manager “What does the manager expect to learn?) • Design and Prep (shape the activities from the objectives and goal/outcomes, timeframe/timeline for the workshop, layout the agenda, gather documents, hold a preview meeting, contact person with subset of review team, coordinate the logistics, share information with other facilitators, personal contact with each participant BEFORE to build trust. Bent has his participants bring three things: 1 – Yourself, 2 – Heart, 3 – Calendar) • Workshop (Frowin and Bent have scripts they will share with the group) 1. Introduction (agenda, logistics, roles, etc.) 2. Goals (from management) 3. Warm up (people have to laugh and Bent does a centering exercise to get the participants thinking about the most impact on you at the feeling level) 4. Project History (hand written on large paper, min 1 ½ hours to 3 hours, to bring out events, capture feelings and see the timeline. This is usually done during the first day of the workshop to also show feelings using a color pen to high light those areas) 5. Topics to work on (This is usually done on the second day of the workshop. Frowin suggested we use an anonymous card to answer the question, “What was not optimal?” then ask a second question “Which has biggest potential for your next project?” Somewhere in the discussions want to talk about the good things!) 6. Sub Groups to work on topics (Give them structure to work on problems with time suggestions: Problem definition, symptoms and reasons, Influences we can’t influence, ideas for solutions, realizations and recognition, concrete action items) 7. Management comes in at the end to see the action items and to add to history 8. Feedback (What do you take out of the workshop? Opens? How did you like the retrospective?) • Feedback & Protocol (Frowin said Protocol is their report – he said they send the report out to only the participants to get feedback, they coach and support the teams for 6 months. They always do a retrospective on the retrospective ) • Closing Activities (scripts and reports go on the server for all the facilitators to see, database – hours, bill, hotel, etc.)
Discussions concerning implementation of the retrospectives:
• Siemens has two full time employees focused on retrospectives. They uses a “network” approach with 25 people who are part-time instructors. These part-time folks need to devote 10-20% of their time towards this effort. • Intel is using an apprentice approach to find other retrospective facilitators. • Use success stories to help with piloting new ideas • How do you decide what projects to turn away if you have more work than you can handle? • It was suggested you hold monthly meetings where you can report on the success of retrospectives to share business information, to schedule future updates. Discuss topics of interest to educate them. •
Discussions concerning sustaining retrospectives:
• Flexible and realize things change • What can we do to not get cutoff? • Management’s support • ROI – 4:1 look at cost, schedule, quality • Participants/project leaders managers keep coming back • Minimizing unhappy participants/managers etc. • Taught people the principles of the retrospective, use a ½ hour to kickoff to explain the concepts • Tell your success stories at management meetings •
Articles
Audiences
on-line newsletter
- friends of the company - customers - interested in compiler and debuggers - interested in software - technical/developers
Quality Week
- conference - SF? - Test focus
Be honest
- hours required - ROI - All the detractors - Doesn’t solve all the problems, not a silver bullet
Benefits
- company - project - individuals
Scenario
- History - Not done well - Topics - Vision – to do better next time - Action items - When it’s not the right tool - Problem couldn’t be solved with retro
When would you not want to do a retro?
- Managers don’t get it - Looking for blame
Things to think about
What do you want the audience to know or feel?
- There’s more to it on many levels: personal, not just canned, more for you. - A tool to improve their work on the project: sample/vision
Ideas for Debra’s presentation
- 5 min project/activity - do a retro on it - short, simple review - get them excited about retro and going back to their org to do it - tried it out myself and got excited - major improvement story - Ask: if you knew at the beginning of the project what you know now, what would have been saved? - Remember to do it at the end.
Targets for articles
- SQE.com/starwest, Oct, Lee Copland likes retro - QAI-USA.com
Retrospective Patterns Thursday, April 12, 2002, Sylvia Beach Hotel, Newport, Oregon Workshop for Retrospective Facilitators
Convener: Ellen; Facilitator: Linda Participants: Gerhard, Pat, Debra, Frowin, Janis, Njke, III, Gundrun, Michael, Esther, Siegie, Bent
Overview to Patterns
Linda led a mini-tutorial for us on patterns.
Patterns are a solution to a recurring problem that’s been successful. This must be a proven solution (not just a good idea, proposal or hypothesis). Must be a proven practice. Thus, “The rule of three” applies: ideally, you should have seen it 3 times in different settings. If you’ve seen something more than once, you are an expert. Thus, we are experts!
Pattern is a way of writing it down. The pattern template or document looks like this:
Pattern: <name of it> Context: <where you would use it*; e.g. use it only in a distributed system; an internal or external facilitator, etc. > Forces: <all the things you have to think about to understand why it’s a hard problem to solve; why is the problem difficult (will skip it today)> Problem: <what is the problem> Solution: <how to solve the problem> Rationale: <may reference research, our own stories; selling the solution> Resulting Context: <what the world will look like after your apply the solution; new problems with arise; consequences – see Alexander’s A Pattern Language for a good idea of how these all connect; also can be named: Consequences, Trade-offs, Side-Effects> Known Uses: <where you saw this; stories; references; 3 new uses> Author: <I wrote it down, even it wasn’t even my idea>
*context: named solutions/patterns like “a stitch in time saves nice” is missing some things like WHEN you’d want to do that stitch and in what other circumstances.
What should we do today? 2 ideas
1. since we’re all retrospective facilitators with varying experience, should we write patterns for retrospective facilitators 2. patterns for successful projects based on our experience
We decided to do both and alternate between the two.
Example of a Pattern
Linda led us to “patlets” (mini-patterns). Mining our experiences for patterns is best done with stories.
Story told by Linda: we used to use food in our internal retrospectives, which lasted an hour. If it was over lunch, we did lunch. But then we weren’t allowed to have food, so we had soda and cookies. Then I was told to skip the cookies (chocolate chip) and soda (Pepsi). I believe that food is important, so I, with my own money, I told my manager that I will bring the cookies. He said, “okay, then with my own money, I’ll bring the soda”.
So, I believe that would be a good pattern.
The group collaborated to create the following patlet from Linda’s story:
Pattern: (possible names): Filling In; Offering; Gift; Provide Offer; Comforting Context: Scarce resources Forces: Problem: obtaining an apparently unavailable recourse; getting something that been take away or never made available in the first place Solution: Fill in a gap; offering/volunteering/contributing to start a trend; start with an offering, take the first step Rationale: when you provide a gift with no strings attached, there is a feeling of needing to give back there is strong research that we have a strong feeling of obligation when given a gift Resulting context: the specific problem goes away; you will be expected to provide them all the time; people don’t take scarcity of resources seriously Known uses: getting food to a group event; American party; pot luck; stone soup (a magic stone is used to make soup, and then ask people to contribute…) Author: person or persons who started to document the pattern; the community also authors
Advice from Linda (Patterns Princess): • keep it as low as possible; otherwise, you’d get a pattern called “Always do the Right Thing” • It’s best to stay low-level, detailed • Save the pattern name for last • Ask yourself: “what are you trying to create”.
Write a summary or abstract for patterns.
Summary: You evolve and document the patterns. You use them to teach new people.
Linda next described the “writer’s workshop”, the next step once the pattern has been drafted. In a relaxed setting, we bring our writing work and give feedback to each other on our work, both positive and corrective. What’s the process to follow that allows us to give feedback while preserving the dignity of the author and the relationship of the community.
We arrive, having read each other’s works. (We come prepared). There is a facilitator, author; readers/reviewers. Each author takes turns in the “hot” seat. You can also adopt people who you might trust (who may not have read the pattern). There are no writer’s workshop police, but typically it’s only the writers.
The author reads the work, a selection of it. Then the writer becomes a fly on the wall. First, they summarize the pattern, then everyone says what they like about the work (some sort of validation). Next, the facilitator asks for “suggestions for improvement” from the group. “I suggest the author…”. The author decides what to do with the comment: keep, use, whatever. Assume the author is the expert.
The facilitator will thank everyone and says “I’m sure the author appreciated it”. Then with a brief pause, may come “the top of the Oreo cookie” – something good the author hears at the end (contributed by a workshop participant).
Finally, you next invite the author back in. she or he doesn’t answer or explain the work, but can ask clarifying questions only. When done, the author is thanked and everyone claps.
In many conferences or workshops, they have a Shepard which tells you if your paper is accepted or not.
In patterns workshops, the program committee gets your pattern and hands it over to a Shepard’s job is to try to work with the author to get it better so their pattern accepted.
Next, we worked in small groups individually to arrive at two sets of patterns at the same time: Those for facilitating retrospectives and those leaned as best practices for projects that we heard during retrospectives. Retrospective Facilitation Patterns note1: first draft; we hope to extend these on the www.retroasis.it Wiki site, once it’s live note2: some of us went for quality while others went for quantity; thus there are more or less pattern elements for these! Note3: where I was unable to read handwriting, I substituted a “?”
PATTERN START WITH DATA Context Group meeting or work session Forces Problem Problem solving or discussion? to a conclusion; gets stuck ? Solution Group has more understand of context and data Rationale Better solution result; the way people think, Satir, ICA Resulting context Known Uses Problem-solving decisions; many meetings Author(s) Esther
PATTERN PEOPLE’S CHOICE Context Retrospectives; dysfunctional team; no one is forthcoming Forces Part-time team involvement, overbearing manager, project on hold Problem What facilitation technique to use to break the silence Solution “Anonymous Agenda”: each person writes 3 ideas for what’s next on the a 3 x 5 card Rationale The threat exist in public utterance; we many surface real problems Resulting context Energy resumes, agenda morphs, maybe retrospective fails Known Uses Problem-solving decisions; many meetings Author(s) ?? (no name on the post (it was pink; you wrote in caps with nice handwriting)
PATTERN GROUP DECISON Context When (some kinds) of conflict arise, let the group decide (sometimes anonymously) ; dysfunctional team; no one is forthcoming Forces Problem Solution Rationale Resulting context Known Uses Author(s) Linda
PATTERN GETTING PRESENT Context Forces Problem They’re mental not yet engaged Solution Get people to “be in the room”; set a context for the group and their work Rationale Resulting context Known Uses Author(s) Diana and Ellen (related patterns) PATTERN GET EVERYONE IN THE ROOM Context Forces Problem Solution A way to assure all are fully present as you begin a retrospective Rationale Resulting context Known Uses Author(s) Pat
PATTERN FEELINGS COUNT Context Forces Problem If you ignore the emotional level you’ll never get to good work Solution Help people to name and honor their emotions and feelings Rationale A shitload of research… Resulting context Known Uses Author(s) Diana and Ellen
PATTERN REVITALIZE THE TEAM Context Forces Problem People get tired, loose focus as time proceeds, get “in the weeds” without reflecting on the purpose of being a project or team (forget the “meta” stuff) Solution Do retrospectives iteratively through the project or at least “mid-point” Rationale Resulting context Known Uses Author(s) Diana and Ellen
PATTERN SEEK THE STORIES Context Forces Problem The facilitator can never know the full story and can be mislead by the sponsor, first person s/he contacts or the biggest whiner. You cannot have the whole picture and don’t have enough context to design the retrospective Solution Do everything you can to find out the stories. This helps the facilitator to stay neutral and have compassion for all the players Rationale Resulting context Known Uses Author(s) Diana and Ellen
PATTERN THE BALANCED FACILITATOR Context Forces Problem The facilitator is not balanced Solution Get enough sleep, wear comfortable clothes, prepare the room to your own needs and wishes, be well prepared. Eat enough but not too much. Rationale Resulting context Facilitator in every kind of meetings Known Uses Author(s) ? (yellow post)
PATTERN DISCUSS UNDISCUSSABLES Context Forces Problem People don’t want to talk about difficult issues or are in denial; people may be inhibited by others knowing things they are not aware of Solution Establish safety, name the undiscussable(s), help them deice what to do with it or them Rationale (related to FEELINGS COUNT) Resulting context Known Uses Author(s) Diana and Ellen
PATTERN SPONSOR “KICK-OFF” Context Forces Problem Solution People take the retrospective (or any facilitated workshop) seriously, they show up, the kick-off motivates participants. Rationale (related to SPONSOR+ SHOW AND TELL Resulting context Known Uses Author(s) Diana and Ellen
PATTERN SPONSOR+ SHOW AND TELL Context Forces Problem Solution Helps the group close; gives participants a sense that their work is taken seriously; gives sponsors accountability and a clue Rationale Resulting context Good follow-up on workshop findings; cynicism is sponsors don’t follow-up Known Uses Author(s) Diana and Ellen
PATTERN PAST, PRESENT AND FUTURE (- OR – COVER THE TIME ZONES) Context Forces Problem Need to understand the past (history, or conditions that impacted the past), present (current conditions) to move toward the future; without understand the past and present, there’ no way to change or improve the future Solution Design the retrospective activities to follow this meta-pattern Rationale Resulting context Known Uses Norm’s book; research and techniques for individual therapy; change models Author(s) Diana and Ellen
PATTERN CIRCLE THE ROOM (-OR- WALKING AROUND) Context Forces Problem Find out if the group or subgroups need help with their activity Solution Move around the room and groups therein, silently, looking at them and their work. See if they are talking about going on vacation, or the content of their group assignment. Rationale Resulting context Known Uses Author(s) Diana and Ellen
PATTERN SELF-REFLECT (-OR- GROUP PROCESS CHECK) Context Forces Problem Groups don’t stop to give themselves feedback on how they’re working together, missing opportunity to make early corrections and improve outcomes Solution Stop and conduct a brief activity or discussion so they can reflect on how they’re doing as a group Rationale Resulting context Known Uses (Self-Reflect “collaboration pattern” written up in Ellen’s book) Author(s) Diana and Ellen
PATTERN EVERYONE ENGAGED (-OR- NOTICE THE BUZZ) Context They are working in dyads or small groups and I don’t want to interrupt Forces Problem How do you find out if your process is working? when they are in the middle of working on an activity, how do I check their progress without interrupting their current group process? Solution Listen for the ‘buzz’ in small groups Rationale Resulting context Known Uses Author(s) Diana and Ellen
PATTERN STEPPING BACK Context Facilitation of a retrospective; convergence on learned lessons or action items Forces Involved stakeholders diversity in roles or preferences; times Problem Chaos perpetuates, convergence is hard Solution Stop and step back instead of push; investigate! On a process level Rationale Pushing will perpetuate chaos; people may feel unheard or group may be too large; missing information Resulting context Adapted process; people feel heard; missing information may get included in the decision making process Known Uses Participatory decisions making (Sam Kaner), Facilitators Toolkit (Lynn Kearney); Quality Volumes (Jerry Weinberg); retrospective experiences (Nynke) Author(s) Nynke
PATTERN WHO DOES WHAT Context Decision-making Forces Problem Unclear who is involved in which part of making a decision Solution Identify who is responsible for getting boundaries, generating options, making the choice, implementing the choice Rationale Resulting context Better acceptance of decisions Known Uses (EG added – note: Decide how to Decide “collaboration pattern” written up in Ellen’s book); Kaner’s decision rules Author(s) Esther
PATTERN HAVE SERIOUS FUN Context They are working in dyads or small groups and I don’t want to interrupt Forces Problem Technical people think fun is bad, yet we do our best work when we are having fun, mentally challenged. Levity and humor is needed to release mental and emotional tension. Solution Use short activities and tools that are allow people to have fun, be silly without being ashamed. Model (as the facilitator) having fun. Rationale Resulting context Known Uses Research on innovation, creativity and play: Schrage (Serious Play), DeKoven (see www.deepfun.com; Czikszentmihalyi (his book, Flow)
Author(s) Ellen
Project Best Practices Patterns (from our experience as retrospective facilitators): Gerhard will sent these out when he can
reto quotes We don't just want to complete a project, we want to learn about completing projects while completing projects. – David Parnas
A man should never be ashamed to own he has been in the wrong, which is but saying, in other words, that he is wiser today than he was yesterday. – Alexander Pope
Retrospective Facilitators’ Gathering Retrospective Friday a.m. April 13, 2002, Sylvia Beach Hotel, Newport, Oregon Workshop for Retrospective Facilitators
Our Retrospective (Agenda; facilitated by Esther)
Create a picture Look for emerging trends Workshop the trends Plan key actions Close
Esther asked these focus questions, which we answered together:
• What are some synonyms or things that come to mind when you think of community? • What are some of your experiences in being in a community? • What has been the significance for you? What has that meant in your life?
The wall showed these trends:
More learning • International/global connection • Writing • Decisions/goals • Synthesis/wholism • Ongoing community • Beyond software • Outreach – data focus • Creativity
Retrospective Slogan Thursday evening, April 12, 2002, Sylvia Beach Hotel, Newport, Oregon Workshop for Retrospective Facilitators Retrospective Slogans: final round
A. On the 7th day, God Retrospected B. Those of Us who Forget history haven’t retrospected yet C. End well to begin well D. If you keep taking the same road, you wind up in the same place E. Slow down – look back – learn from – speed forward F. Insight inside G. This is your project before retrospectives H. History matters I. Experience is a wonderful thing to remember J. The road to failure is paved with lost leanings K. History teaches history teaches L. Déjà vu is good for you M. Reflection is the start of change
Facilitator’s Fav’s: backup Slogans that didn’t make the first cut
• To err is human, to learn is human too • Retroasis • If you don’t have anything nice to say, say it anyway • God said “Let there be light”, and we’ve been reflecting on that ever since • End with the next beginning in mind • The pause that refreshes • Only retrospectives can kill silver bullets
Next Step: Norm will take these and think about them and using them (for example, on retrospectives.com;
Retrospectives: Where to Take Stuff Thursday evening, April 12, 2002, Sylvia Beach Hotel, Newport, Oregon Workshop for Retrospective Facilitators
We listed the vision from the other session
Chart:
In a landmark event, Bent moved us from the “current state” to the the “new current state” with the first retrospective joke, which of course involved a light bulb.
We heard our first poem of retrospectives from Janis!
Groupings of Vision:
Action: • Retrospective facilitators gather/communicate
State:
• There is an established community of retrospective facilitators
Action: • Develop training for retrospective facilitators
State:
• Doing one retrospective a month • I am training retrospective facilitators (we work generatively) • Retrospectives are taught at Devry and ITT • Access to training for retrospective facilitators
Action: • Write articles about retrospective for magazines
State: • Generic articles
Action: • Translate articles for different audiences (also see state Z)
State
• There are at least 10 books on the shelf • Airline magazine articles next to the wine advertisement • There is a listserve or yahoo list
• Infrastructure for parking, patterns, tactics and techniques
Action: • Set up Wiki and build feedback loops and invite you all (and others)
State:
• There is a database of tactics and techniques • We have our own patterns • There is long-term data to support the use of retrospectives
Action: • Facilitate retrospective for community orgs • Translate articles for different audiences
State Z:
• Pattern “songs” from retrospectives • They are retrospective jokes • Scott Adams publishes a cartoon that makes fun of the pointy hair boss that doesn’t want to do them • There is a prominent flagship project we can talk about, that is referenced and discussed as a ‘biggie’ example of the success of retrospectives that can be talked about • Organizations are telling stories of success and benefits • Retrospective poems • Retrospective as a term becomes a common, household
State:
• I don’t have to convince people, just a light nudge • Many Mini retrospectives are woven throughout the project • It’s an accepted part of development practice • People understand the linkage to other practices • Retrospectives are part of the standard toolbox for introducing change and are linked to other techniques that do that (e.g. the hot methodologies like today’s XP, Scrum, etc. have retrospectives as part of their process) • Managers are confident in their use because it’s a standard practice worldwide • People will plan doing a retrospective when they start a project; they ask for it when they begin a project • Results of retrospectives change daily project activity
Action:
• Get repeat business
State: • retrospective facilitaors are familiar faces in the shops that hire them
Action:
• exploit our success
State:
• It’s an accepted par to of the development process • Etc.
Ultimate State:
• There is a prominent flagship… • 10 books on the shelf • doing 1 per month • results change daily project activity
Next step:
Michael will put the visual model we build on the floor into Visio, cross check the actions and states documented here and we’ll want to post this on retroaosis.it.
ellen pattern:
**Name** Self-Reflect Context A group is working together to deliver one or more products over a period of time. The members must continue to work both together and separately to produce the product. **Problem** Group members often meet and work together without stopping to evaluate their own behavior—helpful and harmful—as a group. They may miss opportunities to learn what is working and what should be changed in order to improve both their product and their productivity. How do you get a group to quickly and effectively generate meaningful information that reinforces, helps people to improve, and leads to commitment for action? **Solution** • Always take time at the end of each collaborative event to assess how effective a job the group did.
• Make sure that all comments are treated as valid. • Visibly record comments as they are presented. • Seek commitment for new and reinforcing behaviors.